Figure 1. An m=3 strategy. 
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Figure 2: N=101 s=1 
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Figure 3: N=101 s=2 
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Figure 4a: m=4, s=2, N=101 
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Figure 4b: m=4, s=2, N=101 
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Figure 5: m=6, s=2, N=101 



0.9 



0.8 



0.7 



,0.6 



to 

CD 
Q. 



0.5 
^0.4 



0.3 



0.2 



0.1 



d 



10 



20 



30 

m-bit past 



40 



50 



60 



45 




47 



Figures: N=101 s=6 
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Figure 11. 
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Figure 12a: m=4, s=6, N=101 
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Figure 12b: m=4, s=6, N=101 
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Figure 14. mutual information, N=101 
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Figure 15a, Segment of time series of L^, for m=2, s=2, N=101 
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Figure 15b, Segment of time series of L^, for m=8, s=2, N=101 
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Figure 20. An m=3.46 strategy. 



m-string 


Prediction 


0000 





0010 


1 


*011 


1 


*110 





0111 





1000 


1 


*001 





1010 


1 


*100 





*101 





1111 


1 



60 



Figure 21a. s=2, N=101 
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Figure 21b. s=2, N=101 
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Figure 22. s=2, N=101 
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Figure 23. vs. m, s=2 N=101. 
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Figure 24. s=2, N=101 
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Figure 25. s=2, N=101 
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Figure 26a. N=101, s=2, m=3 



Figure 26b. N=101, s=2, m=3 
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Figure 26c. N=101, s=2, m=4 
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Figure 26e. N=101, s=2, m=5 
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Figure 26d. N=101, s=2, m=4 
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Figure 26f. N=101, s=2, m=5 
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Figure 26g. N=101, s=2, m=6 



Figure 26li. N=101, s=2, m=6 
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Figure 26i. N=101, s=2, m=7 



4.5 



wealth 



X 10 



5.5 

4 



Figure 26k. N=101, s=2, m=8 
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Figure 261. N=101, s=2, m=8 
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The Structure of Adaptive Competition in Minority Games 



Radu Manuca+, Yi Li, Rick Riolo and Robert Savit* 
Program for the Study of Complex Systems and Physics Department 

University of Michigan 
Ann Arbor MI 48109 

Abstract 

In this paper we present results and analyses of a class of games in which heterogeneous 
agents are rewarded for being in a minority group. Each agent possesses a number of 
fixed strategies each of which are predictors of the next minority group. The strategies 
use a set of aggregate, publicly available information (reflecting the agents' collective 
previous decisions) to make their predictions. An agent chooses which group to join at a 
given moment by using one of his strategies. These games are adaptive in that agents can 
choose, at different points of the game, to exercise different strategies in making their 
choice of which group to join. The games are not evolutionary in that the agents' 
strategies are fixed at the beginning of the game. We find, rather generally, that such 
systems evidence a phase change from a maladaptive, informationally efficient phase in 
which the system performs poorly at generating resources, to an inefficient phase in 
which there is an emergent cooperation among the agents, and the system more 
effectively generates resources. The best emergent coordination is achieved in a 
transition region between these two phases. This transition occurs when the dimension of 
the strategy space is of the order of the number of agents playing the game. We present 
explanations for this general behavior, based in part on an information theoretic analysis 
of the system and its publicly available information. We also propose a mean-field like 
model of the game which is most accurate in the maladaptive, efficient phase. In 
addition, we show that the best individual agent performance in the two different phases 
is achieved by sets of strategies with markedly different characteristics. We discuss 
implications of our results for various aspects of the study of complex adaptive systems. 

* To whom correspondence should be addressed, e-mail: savit@umich.edu. 

+ Current address: College of William and Mary, Physics Department, Williamsburg, VA 

23187-8795 
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I. Introduction 

Complex adaptive systems of agents with co-adaptive or co-evolving strategies are 
systems of agents which compete for some resource, or set of resources, and by their 
competition alter their environment. In such systems agents are tj^ically endowed with 
heterogeneous strategies, beliefs and behaviors. The collective action of these agents in 
general alters their environment, and so, as time goes by, agents must reevaluate and 
possibly change their strategies to more effectively compete in the altered environment. 
Markets, ecologies, political structures and competition among businesses are all 
examples of such systems. Indeed, it is difficult to imagine an interesting system in the 
biological or social sciences which does not encompass elements of co-evolving 
competitive systems. 

Despite their central role, not much is understood about the nature or behavior of such 
systems in general. Although many such systems have been studied by a variety of 
methods, and much has been learned about those specific systems, there is currently no 
general theory of N agent competitive systems with co-evolving strategies. In fact, there 
is not even a well-established epistemology. We do not know, in general, what range of 
behaviors such systems can exhibit, nor under what general circumstances various 
behaviors will be manifest. In fact, we do not even know what the sensible questions are 
(by which we mean questions that can have sensible, well-defined answers) which we can 
ask about such systems. 

This latter comment may require some explanation. Depending on the nature of the 
djaiamics, there are well-known limitations to questions that can be reasonably asked of 
various physical and mathematical systems. We know, for example, that if a system is 
chaotic, it is not sensible to ask for a precise prediction of its behavior over a long time. 
As another example, we know that in quantum mechanics one cannot sensibly ask for a 
precise, simultaneous determination of the position and momentum of a particle. Given 
the general dynamical architecture of N-agent complex adaptive systems with co- 
evolving strategies, we do not know if there are analogous limitations (and if so, what 
they are), to the questions we can sensibly ask of these systems. 

One common approach to trying to understand the behavior of N-agent complex adaptive 
systems with co-evolving strategies is driven by the specific disciplinary interests of 
researchers. In this approach, one tries to model specific systems with some degree of 
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realism. While the study of such models may indeed illuminate the object of their study, 
there are problems with this approach: For example, with many mechanisms 
incorporated ab initio into a model, it is difficult to understand precisely which aspects of 
the model are responsible for various results. Here the epistemological problem also 
becomes important. It may be, for example, that some of the results gleaned from very 
specific models may not be robust or meaningful, casting doubt on inferences drawn from 
these results. 

A different approach, that we find more useful, is to study in detail the simplest possible 
models that capture at least some of the most basic underlying dynamics that may appear 
in a wide range of N-agent co-evolving systems. Such very simple models, while not 
necessarily accurately reflecting the ingredients of any specific system, have the virtue of 
being controllable and potentially understandable. These models can help us address the 
most basic issues of complex adaptive systems giving us great insight, not only into what 
emergent behaviors we can expect from various fundamental underlying djoiamics, but 
also what the appropriate questions are that we can ask of such systems. 

In this paper, we analyze one such simple (and we believe, paradigmatic) system. This 
system is a repeated game of N players, choosing (independently) according to some set 
of strategies to join one of two groups. At each time step only members of the minority 
group are rewarded. As explained in more detail below, this system has a most 
remarkable behavior.^ The system manifests a phase change from a phase which is in a 
certain sense, efficient, but maladaptive, to a phase which is, in a certain sense, 
inefficient, but in which the system as a whole utilizes it's resources effectively. As one 
increases the size of the strategy space available to the agents, one moves from the former 
to the latter phase. The dynamics whereby this transition takes place are subtle and 
fascinating, and are the result of an emergent coordination in the decisions taken by the 
agents. This coordination is mediated only by aggregate, publicly available information, 
and does not result from any detailed knowledge of the behavior of one agent by any 
other. The system also has some intriguing properties reminiscent of a spin-glass in 
statistical mechanics. 



' R. Savit, R. Manuca and R. Riolo, University of Michigan Program for the Study of Complex Systems 
preprint no. 97-12-001. (1997). Available at http://xxx.lanl.gov/abs/adap-org/9712006. Submitted for 
publication. 
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In the next section we define the model, and offer some simple interpretations to motivate 
its relevance. In section HI we present the results of computer experiments performed 
with this model. In particular, we will demonstrate that the model, despite its incredible 
simplicity, has a remarkable and surprising richness of behaviors. In section IV we will 
study the results of section HI with an eye toward understanding the general system-level 
behavior of these systems. In section V, we briefly touch on some aspects of individual 
agent wealth accumulation in these games. In particular, we study what properties of the 
agent's strategies are important for accumulating wealth. In section VI, we will present a 
mean-field like approximation to the djaiamics of the model which will allow us to 
explain the behavior of the model in some of the parameter space. The mean-field 
approximation, will also help elucidate the nature of the djmamics in parameter regimes 
where the approximation breaks down. The paper ends with section Vn, in which we 
present a summary and discussion of our work. 

II. The Model and its Motivations 

The simple model of competition we discuss here consists of N agents playing a game . 
The rules of the game are as follows: At each time step of the game, each of the N agents 
playing the game joins one of two groups, labeled or 1. Each agent that is in the 
minority group at that time step is awarded a point, while each agent belonging to the 
majority group gets nothing. An agent chooses which group to join at a given time step 
based on the prediction of a strategy. (In general, different agents use different strategies, 
as we shall explain below.) A strategy of memory m is a table of 2™ rows and 2 columns. 
The left column contains all 2™ combinations of m O's and I's, and each entry in the right 
column is a or a 1. To use a strategy of memory m, an agent observes which were the 
minority groups during the last m time steps of the game and finds that entry in the left 
column of the strategy. The corresponding entry in the right column contains that 
strategy's prediction of which group (0 or 1) will be the minority group during the current 
time step. An example of an m=3 strategy is shown in Fig. 1. Thus, a strategy uses 
information from the historical record of which group was the minority group as a 
function of time, which is the information publicly available to all the agents. 

In each of the games discussed here, all strategies used by all the agents have the same 
value of m. At the beginning of the game each agent is randomly assigned s (generally 



^ D. Challet and Y.-C. Zhang, Physica A, 246, 407 (1997). A related model has also been considered by N. 
Johnson, S. Jarvis, R. Jonson, P. Cheung, Y. Kwong and P. Hui, Physica A 256, 230 (1998). 



5 



greater than one) of the 2 possible strategies of memory m, with replacement. For his 
current play the agent chooses from among his strategies the one that would have had the 
best performance over the history of the game up to that time. Ties among an agent's 
strategies can be decided either by a coin toss, or by choosing to play the strategy that was 
most recently played. Following each round of decisions, the cumulative performance of 
each of the agent's strategies is updated by comparing each strategy's latest prediction 
with the current minority group. Because the agents each have more than one strategy, 
the game is adaptive in that agents can choose to play different strategies at different 
moments of the game in response to changes in their environment; that is, in response to 
new entries in the time series of minority groups as the game proceeds. Because the 
environment (i.e. the time series of minority groups) is created by the collective action of 
the agents themselves, this system has very strong feedback. 

Although this system is adaptive, the versions we analyze here are not, strictly speaking, 
evolutionary. The strategies do not evolve during the game, and the agents play with the 
same s strategies they were assigned at the beginning of the game. Evolutionary 
mechanisms can be simply incorporated into this model, and will be discussed elsewhere. 

While this model is very simple, and far from a realistic model of any particular 
biological or social system, it is not difficult to see the relevance of the djaiamics 
incorporated in the model. First, the model was inspired, in part, by the El Farol 
problem^. El Farol is a bar in Santa Fe, New Mexico that plays Irish folk music on 
Thursday nights. Many people want to go to the bar to hear the music, but only if there 
are fewer than a certain number of people at the bar. Otherwise, there will be too much 
noise, and the patrons will be unable to enjoy the music. In our model, we can let 
represent attendance at the bar, while 1 means staying home. Then, one benefits (and is 
awarded a point) if one attends the bar on a night when fewer than half the agents attend. 
One also benefits (and is awarded a point) if one stays home on a night when more than 
half the agents attend the bar.^ 



^ The latter is the default for our games, but, as we shall discuss, the system-wide behavior does not 
generally depend on whether tie-breaking is achieved stochastically or deterministically. 

W. Brian Arthur, Amer. Econ. Assoc. Papers and Proc. 84, 406 (1994). 
^ We are saddened to report that a fire has taken a serious toll on this fabled establishment. However, 
reUable first hand accounts indicate that it is again operational. (E. Bonabeau, private communication). 
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This game can also be interpreted as a toy model of a market. In this market, each agent 
has an indefinite supply of money and widgets. At each time step, each agent must either 
buy or sell one widget. Call group the buyers and group 1 the sellers. The price of the 
widgets can take on only two values, and is determined by a simple supply-demand rule: 
If there are more buyers than sellers, the price is high, and if there are more sellers than 
buyers, the price is low. If group is in the minority, then there are more sellers than 
buyers, the widget price is low, and the buyers (the minority group, 0) do well, in that 
they buy at a low price. Therefore, they each get a point. If group 1 is in the minority, 
then there are more buyers than sellers, the widget price is high, and the sellers (the 
minority group, 1) do well, in that they sell at a high price. Therefore, they are awarded a 
point. 

From these two interpretations it is clear that the dj^amics associated with minority 
membership plays an important role in adaptive competitive systems. Other examples of 
systems that are, at least in part, driven by minority-membership djniamics include 
vehicular traffic on roads, in which each agent would prefer to be on an uncongested 
road,^ packet traffic in networks, in which each packet will travel faster through lesser 
used routers, and ecologies of animals looking for food, in which individuals do best 

o 

when they can find areas with few competitors . Of course, for any particular system, 
care must be taken to understand how minority dynamics is to be imposed, and what 
other ingredients may be necessary. Nevertheless, studying minority dj^amics is clearly 
of great interest for understanding the fundamental general behavior of adaptive 
competitive systems. 

III. Basic Results 

In this section, we will present a set of system-level results which demonstrate a 
remarkable and rich phase structure for minority games. We will first describe the 
general methods used for our experiments. We will then describe basic results for (a) one 
strategy per agent, (b) two strategies per agent, and (c) more than two strategies per agent. 



*K. Nagel, S. Rasmussen, and C. Barrett, Network Traffic as a Self-Organized Critical Phenomena in Self- 
organization of Complex Structures: From Individual to Collective Dynamics, p. 579, F. Schweitzer (ed.) 
(Gordon and Breach, London. 1997). 

' Coordination of the Internet, B. Kahin and J. Keller (eds.) (MIT Pres, Cambridge MA, 1977); B. 
Huberman and R. Lukose, Science, 111, 535 (1997). 

* See, for example. Ecology and Evolution of Communities, M. L. Cody and J. M.Diamond (eds) (Harvard 
University Press), Cambridge MA, 1975); Individual-Based Models and Approaches in Ecology : 
Populations, Communities, and Ecosystems, D. DeAngehs and L. Gross (eds), (Chapman & Hall, New 
York, 1992). 
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Note that in this section and in most of the rest of this paper, we will primarily present 
and discuss experiments done with integer m. However, it is easy to generalize to non- 
integer m, and, in Section IVB, we shall present and discuss some results for non-integer 
m. The general case of non-integer m, which also has other implications, will be 
discussed more extensively in a forthcoming publication.^ 

Unless stated otherwise, the experiments we describe were performed with the following 
default settings: Generally, for each value of m, N and s, 32 different runs were 
performed. A different random distribution of strategies to the agents was assigned for 
each run. In addition, a random initial history of m+1 minority groups was necessary for 
the initial evaluation of the agents' strategies. ^'^ In most figures in which a single value of 
a quantity is presented for a given m, N and s, that quantity is an average over 32 runs. 
For most of our runs, in which N ranged between 1 1 and 1001, we found that 10,000 time 
steps were generally enough to sufficiently eliminate transients for the purpose of 
estimating quantities. The largest values of N, however, required runs of 100,000 time 
steps. Typically results were based on the final 20%-30% of data from each run. The 
reason for these fairly long runs is discussed in subsection B, below. 

A. One strategy per agent, s=l 

To begin, we consider the case of only one strategy per agent. In this exceptional case, 
there is no adaptivity. The response of each agent to a given string is always the same. 
Because of this, it is not difficult to see that the time series of minority groups will 
eventually become periodic with a period that depends on the particular (random) 
distribution of strategies to the agents. 

An important quantity for us will be the standard deviation, o, of the time series of the 
number of agents belonging to group 1 (=Lj). (This information is not available to the 

agents but it is available to the researchers.) The mean of this series is generally close to 
(but slightly less than) 50% for all values of N, m and s (we shall return to this point 
below), and so the standard deviation, a, of this time series is a measure of how well the 
commons do: The smaller o, the more total points are awarded to all agents combined. 
That is, if there are typically many fewer than 50% of the agents in the minority, then 

^ Y. Li, R. Manuca, R. Riolo and R. Savit, in preparation. 

The overall statistical properties of each run do not depend on the initial history of minority groups. 
However, more detailed aspects of the results may. For example, under some circumstances the relative 
wealth of a specific agent may depend on this initial history. This will be discussed in more detail in Ref. 9, 
above. 
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a will be large and there will be few total points awarded. On the other hand, if a is 
small, then most of the time the minority group will consist of only slightly fewer than 
half of the agents, and more total points will be awarded. 

In Fig. 2, we plot a for a number of experiments with s=l, N=101 and various values of 
m. The horizontal line on this graph is the value that o would have if each agent made 
his choice randomly and independently (with equal probabilities for and 1) at each time 
step. We call this the random choice game (RCG). In this figure we note that there is a 
wide dispersion in a for a given m, but that the mean of the a's is near the RCG result. 
This is not surprising. Each run of the game effectively represents a small temporal 
sample, (the duration of which is equal to the period of that particular game, determined 
by the initial distribution of strategies) of a random game. Thus, we expect, in general, 
some dispersion about the RCG result, which is what we see. This s=l case is trivial, but 
provides a good comparison for the considerably more interesting games with s>2. 

R Two strategies p er agent, s=2 

Consider next the case s=2 which is the simplest case of adaptivity. The reason that the 
game must tj^ically be run for thousands of time steps is that the agents must have 
sufficient time to "learn" about each other through the collective, publicly available 
information of the time series of minority groups. Only with this learning is it possible 
for the agents to coordinate their strategy choices. This was not an issue in the s=l 
experiments in which there was no learning, and no possibility of agents coordinating 
their choices. 

To begin to understand the behavior of this system, consider, as before, o, the standard 
deviation of the time series of the number of agents Li. The behavior of o is quite 
remarkable. In Fig. 3, we plot a for these time series as a function of m for N=101 and 
s=2. For each value of m, 32 independent runs with different initial distributions of 
strategies were performed. The horizontal dashed line in this graph is at the value of a 
for the random choice game (RCG), i.e. for the game in which all agents randomly choose 
or 1, independently and with equal probability at each time step. The comparison with 
Fig. 2 is startling. 



" Although the results shown in Fig. 2 are spread near the result of the RCG, it is clear that there is a bias 
toward smaller a. This is just a trivial consequence of the fact that these games have a finite periodic 
structure in which Li is also a periodic function. 
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Note the following features: 

1. For small m, the average value of a is very large (much larger than in the random 
case). In addition, for m<6 there is a large spread in the o's for different runs with 
different (random) initial distributions of strategies to the agents, but with the same m. 

2. There is a minimum in o at m=6 at which o is less than the standard deviation of 
the random game. We shall refer to the value of m at which the o vs. m curve (for fixed 

12 

N) has its minimum as mg. Thus, Fig. 3, indicates 6 as an approximation to mc. Also, 
for m>mc, the spread in the a's appears to be small relative to the spread for m<mc. 

3. As m increases beyond 6, a slowly increases, and for large m approaches the 
value for the random choice game. 

The system clearly behaves in a qualitatively different way for small and large m. To 
further study the dynamics in these two regions, we consider the time series of minority 
groups, (=G), the data publicly available to the agents. We want to study the information 
content of strings of consecutive elements of this time series of various lengths (including 
strings of length m) for different values of m and N. Since the strategies use only the 
information contained in the most recent m-time steps of G, it is natural to ask how much 
information is accessible to the strategies. To do this, we consider the conditional 
probability P(lluk). This is the conditional probability to have a 1 immediately following 
some specific string, Uk, of k elements of G. For example, P( 110 100) is the probability 
that 1 will be the minority group at some time, given that minority groups for the four 
previous times were 0,1,0 and 0, in that order. Recall that in a game played with memory 
m, the strategies use only the information encoded in strings of length m to make their 
choices. In Fig. 4, we plot P(lluk) for G, the time series of minority groups generated by a 
game with m=4, N=101 and s=2. Fig. 4a shows the histogram for k=m=4 and Fig. 4b 
shows the histogram for k=5. Note that the histogram is flat at 0.5 in Fig. 4a, but, 
remarkably, is not flat in Fig. 4b. Thus, any agent using strategies with memory (less than 
or) equal to 4, will find that those strings of minority groups contain no predictive 
information about which group will be the minority at the next time step. But recall that 



In general, ni;. is not an integer. Since this experiment was performed only for integer m, 6 must be 
considered an estimate of m,,- In section IVB we will show how to interpolate to non-integer m, and we will 
see that for N=101 m^ is approximately in the range 5.2-5.7. In the remainder of this paper, values of m^ 
deduced from experiments using only integer m, should be understood to be rough approximations to noc. 
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this time-series was itself generated by agents playing strategies with m=4. Therefore, in 
this sense, the market is efficient (at least with respect to the strategies) and no strategy 
playing with memory (less than or) equal to 4 can, over the long run, accumulate more 
points than would be accumulated randomly^^. But note also that the time series of 
minority groups is not a random (IID) string. There is information in this string, as 
indicated by the fact that the histogram in Fig. 4b is not flat. However, that information is 
not available to the strategies used by the agents in playing the m=4 game which 
collectively generated that string in the first place. The histogram for k=m is flat for m< 
mc (although it does begin to show some small 0(T ) fluctuations for m just below mc). 

We can repeat this analysis for m>6 (N=101, s=2). For this range of m, the 
corresponding histogram for k=m is not flat, as we see in Fig. 5 for the m=6 game. In this 
case, there is significant information available to the strategies of memory m and the 
"market" is not informationally efficient with respect to the strategies. Indeed, some 
individual agents using their strategies do accumulate significantly more points than they 
would by simply making random guesses. However, for m<5, no agent ever achieves 
more than 50% of the possible available points. The wealth distribution of the agents in 
the two phases is very interesting and will be discussed in section V, below, and in detail 
in forthcoming publication^. 

How does the system behavior change as we change the number of agents? One can 
repeat the calculation of a for different N, again with s=2. One finds, plotting a vs. m for 



Actually, the structure in this phase is a little more subtle. The system is informationally efficient with 
respect to the strategies, but is not efficient with respect to the agents. In fact, there are strong correlations 
between the minority groups over long times of order 2" time steps, which results in the very poor 
performance of the system in this regime. This will be discussed in more detail in section IV A, below. 
Note also that these long time correlations are not to be confused with the non-IID conditional probabihties 
(that indicate information over time scales of order m+1) discussed in the text and illustrated, for example, 
in Fig. 4b. Despite these subtleties this phase does partake of the usual idea that efficiency implies an 
absence of usable information. As we shall see below when we study the inefficient phase, the use of the 
idea of efficiency is quite appropriate in this context. For a discussion of the idea of efficient markets as 
traditionally understood in economics, see E. F. Fama, J. Finance, 25(2), 383 (1970) and E. F. Fama, J. 
Finance, 46(5), 1575 (1991). 

In fact, asymptotically, for m<mc, no strategy can be right more than 50% of the time. If the behavior for 

these values of m were truly random, there would be fluctuations of order T^l^ in the number of times a 
strategy won over a time T. But in our case, these Gaussian fluctuations are absent, and strategies are 
limited to being correct no more than 50% of the time. A corollary to this is that the histogram in Fig. 4a is 
extremely flat, and does not contain even the random fluctuations that would be present for an IID series. 
The reason for this will be explained below. 

Note that the particular shape of the histogram in Fig. 5 as well as in Fig. 4b depends on the initial 
distribution of strategies to the agents. But in all cases with m>mc or with mKm^ and k>m, the histograms 
do indicate the presence of significant predictive information. 
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fixed N, that in all cases one obtains a graph similar to that in Fig. 3, but in which the 
position of the minimum, mc, is proportional to InN. In addition, a and the spread in a 
behave in very simple ways with changes in N which differ depending on whether m is 
greater than or less than mc. In Fig. 6 we study the behavior of a as a function of N for 
m=3 and m=16. For the range of values of N used in these figures, m=3 is to the left of 
the minimum in the curve of a vs. m (i.e. 3< mg for all these values of N) and m=16 is to 
the right of the minimum (16> mc). In Fig. 6a we plot o vs. N on a log-log scale. We see 
that for m=3 a is proportional to N, while for m=16, a is proportional to N^^^. This is 
tj^ical: For fixed m, and integer m< mc, o is proportional to N, while for fixed m and 
m> mc a is proportional to N^^^ Fig. 6b, we plot, again for m=3 and 16, Aa/a as a 
function of N, on a log-log scale . Here Aa is the standard deviation of the o 's for runs 
with the same value of m, N and s, but different initial (random) distributions of 
strategies. The horizontal lines indicate that the dependence of Ao on N for fixed m is the 
same as that of a, namely for m=3 Aa is proportional to N, while for m=16, Aa is 
proportional to N^^^. As before, this behavior is representative of the two behaviors seen 
for values of m< mc and values of m> mc, respectively. 

The transition between these very different behaviors is at mc~lnN. To see this explicitly, 
we plot, in Fig. 7, a^/N as a function of z=2'^fN on a log-log scale for various N and m 
(with s=2). We see first that all the data fall on a universal curve. (As we shall discuss in 
section VI, we have also found, using mean-field-like arguments that are most accurate in 
the low m phase, that for fixed s, a^/N is a function only of z.) The minimum of this 
curve is near 2riic/N=Zc=0.5, and separates the two different phases. The slope for z<Zc 
approaches -1 for small z, while the slope for z>Zc approaches zero for large z, consistent 
with the results of Fig. 6. Because a^/N depends only on z, it is clear that for fixed z, a is 
proportional to N^'^ for any fixed z, both above and below Zc. In addition, it can be shown 
that, for fixed z, Aa is approximately independent of N, approaching a z-dependent 
constant as N— ><». The N— ><» limit of Aa is large for small values of z and appears to 
decrease monotonically with increasing z. It is unclear whether or not Aa is non analytic 
at Zc. 

C More than two strategies p er agent, s>2 



As before, since these experiments involve only integer m, this value of should be considered only an 
estimate. Experiments with non-integer m indicate a value of closer to 1/3. See section IVB, below. 
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We now study the case in which each agent has more than two strategies among which to 
choose. In Fig. 8, we present a plot like that of Fig. 3, but for the case s=6. We see here 
that the qualitative features seen in Fig. 3 persist: o is large for small values of m 
(m<mc), first decreases as m increases, reaches a minimum at some value of m, then 
increases, and approaches, asymptotically, the result of the RCG. There is also a spread 
in o, Aa, which has the same qualitative behavior as in the s=2 case, namely, for m<mc 
Ao and o are proportional to N, while for m> mc Aa and o, are proportional to N^^^, as 
demonstrated in Fig. 9. For s=6, there is also a scaling result similar to that shown in Fig. 
7 for the s=2 case. In Fig. 10, we plot, for s=6, the mean value of o^/N as a function of z 
for various values of m and N. As in Fig. 7, we see that all the points lie on a universal 
curve. Notice, however, two important differences with the s=2 case: First, the dip in 
Fig. 10 is not as deep as the corresponding dip in Fig. 7 for the s=2 case. Second, the 
position of the minimum is slightly offset from the value for the s=2 case. In Fig. 1 1 we 
plot, these universal scaling curves for various values of s on a log-log plot. Here the 
horizontal axis is ^=[z/zc(s)], so that the minima of all the curves are at ^=1. For large 

and small values of ^, the curves coincide, approaching the RCG result for large ^, and 
falling on a curve with a slope of -1 for small ^. However, for intermediate values of ^, 
there are differences. As s increases, the value of Zg (i.e., the value of z at which the 
curve has its minimum) appears to drifts slightly to somewhat larger values of z as can be 
seen in Fig. 11 and also by comparing Figs. 7 and 10. Even more importantly, the dip at 
Zc becomes increasingly shallow, and for very large s, the universal curve approaches two 
straight lines, one with a slope of -1 (small z), and the other with a slope of at the value 
of the RCG. These two lines intersect at ^ =1- 

The information content above and below Zc for s>2 is similar to that for the s=2 case. In 
Fig. 12, we present P(lluk) for G, the time series of minority groups generated by a game 
with m=4, N=101 and s=6. Fig. 12a shows the histogram for k=m=4 and Fig. 12b shows 
the histogram for k=5. As in Fig. 4, the histogram is flat in Fig. 12a, but not flat in Fig. 
12b, showing that there is no information available to the strategies playing the m=4 
game with N=101 and s=6, but that the sequence, G, is not ED. Similarly, in Fig. 13, we 
plot P(lluk) for k=6 generated in the m=7 game with s=6. As in Fig. 5, this histogram is 
not flat, demonstrating that there is information available to the strategies in the z>Zc(s) 
phase, even when s>2. 
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Another very convenient way to display the information content that is accessible to the 
strategies in the time series of minority groups, is to compute the mutual information 
between the current minority group and the sequence of the last m minority groups. The 
mutual information between two data sets can be thought of as a measure of the extent to 
which there is common information between the two sets. For the game played with 
memory m, we consider the mutual information defined as 

Q(m)=H(m) + H(l) - H(m+1) (3.1) 

where H(j) is the entropy of the distribution of strings of length j. In Fig. 14 we plot 
Q(m) as a function of ^ on a semi-log plot for N=101 and s=2 and 16. In both cases, Q=0 
for C,<1, and Q>0 for ^>1. Note also that for positive ^, Q is much larger for s=2 than for 
s=16. This is an indication that, for a given m and N, the strategies have more 
information available to them for smaller s. It is also related to the observation that the 
dip in Fig. 1 1 is deeper for smaller s. 

IV. Explication of the Phase Structiire 

In this section we will examine in more detail the behavior of the system in three regions: 
(A) the low m, strategy-efficient phase, (B) the transition region, and (C) the 
informationally inefficient region. We will present explanations for the behavior of the 
system in these regions, relying on the results of section HI, as well as on some more 
detailed results to be presented in this section. Topics that will be covered include an 
explanation of the poor system-wide performance and strategy-efficiency in the low-m 
phase, qualitative arguments for the position of the phase transition, a more detailed look 
at the transition region, and a discussion of the degradation of system performance as m 
increases beyond mg. In addition, some of these topics will be further elucidated when we 
examine some aspects of individual agent wealth, in section V. In subsection B, we will 
introduce a method of interpolating to non-integer values of m, for the purpose of 
studying the transition region in greater detail. 

A. Dynamics in the efficient phase. 

To begin to understand our system, let us first look more closely at the dynamics in the 
efficient (low m) phase. In Fig. 15 we show a short segment of the time series. Figure 



Time Series Prediction: Forecasting the future and Understanding the Past, A Weigend and N. 
Gershenfeld (eds.) (Addison- Wesley, Reading, MA, 1994), and references therein. 
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15a shows a segment for N=101, s=2 and m=2, which is in the efficient phase. Fig. 15b 
shows a segment for N=101, s=2 and m=8, which is in the inefficient phase. For 
comparison, the horizontal dashed lines indicate the range of one standard deviation for 
N=101 in the RCG. The first thing we notice is that Fig. 15a shows a bursting structure 
with segments of large variation separated by segments of smaller variation, a feature 
which is absent in Fig. 15b. In Fig. 15a we have further marked, by open circles, those 
times at which the last two minority groups were 01. We note that the odd occurrences of 
this string elicit responses from the agents that result in relatively large minority groups 
(i.e. small deviations in the population of group 1 from 50%), while for even occurrences 
there are large deviations in Lj. 

We can understand this as follows: When m is relatively small, then there is a reasonable 
probability that an agent will have two strategies that are fairly similar. Suppose, for 
example, that two m=2 strategies differ in only one (say the 01) entry, and are otherwise 
the same. The first time the sequence 01 appears in G, either strategy a priori may be 
chosen, with equal probability to be played by its agent, and so we expect that the 
population of group 1 will be close to 50% of the total number of agents. Suppose that 
the minority group happens to be group 1. If an agent has two strategies which differ only 
in the 01 entry, then the strategy that predicts 1 following an occurrence of 01 in G will 
have one extra point. Now suppose the game continues. Up until the next occurrence of 
01 in G, both strategies will accumulate the same number of points. At the next 
occurrence of 01 in G, the strategy that predicted 1 for the previous occurrence of 01 will 
have an extra point, and so the agent will play that strategy. If there are a significant 
number of agents which have strategies that are similar (aside from their response to the 
string 01), then there will be a large population in group 1 at this occurrence of 01. 
Consequently, the minority group will be 0, and the two strategies that differ only in their 
responses to 01 will again have an equal number of points. Thus, at the next occurrence 
of 01, the population of the minority group will once again be relatively close to 50% and 
the cycle will continue. That is, odd occurrences of a given string produce a minority 
population which is close to 50% (and consequently a small deviation in Lj from the 
mean), while even occurrences produce a large deviation in Lj from the mean, and a 

minority group opposite that of the preceding odd occurrence. It is the responses of the 
system to the even occurrences of strings that give rise to the large deviations in Lj, and 

are responsible for the fact that in the efficient, small m phase, a (as well as Aa) is 
proportional to N for fixed m. Thus, the typical choices of the agents may be said to be 
maladaptive, in that their choices lead, half of the time (at the even occurrences of a given 
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string) to very small minority groups, and consequently to a poor utilization of the 
resources. This is a clear manifestation of herd following or faddish behavior often seen 
in social systems. 

This dynamics also explains the flat conditional probability distribution of Fig. 4a. 
Alternating occurrences of a given string produce opposite responses in the sequence of 
minority groups. Thus, at any given moment in a game, the number of times a 1 follows 
a given m-string of minority groups will differ by at most one from the number of times 
that same m-string is followed by a 0. Consequently the conditional probabilities will be 
very close to 0.5 for all m-strings. 

Notice that because of the way in which the agents evaluate their strategies, the djaiamics 
in this phase induces a new time scale. The time scale over which the strategies operate 
is m~i.e. each strategy looks back only m time steps to produce its prediction. But the 
agents evaluate their strategies from the beginning of the game, and so the djmamics 
produce a time scale of order 2™ (i.e., the tj^ical time between occurrences of the same 
string) which is the time scale relevant for the adaptive dynamics. 

Qualitatively, this period-two dynamics persists as long as there is a reasonable 
probability that the relative ranking of an agent's two strategies will not be altered 
between successive occurrences of a given string. Clearly, as m increases it becomes 
increasingly likely that the relatively rankings of an agent's strategies will be altered, since 
the time between successive occurrences of the same string (~2™) increases. 
Consequently, we expect that the bursty structure in Lj and the maladaptivity will become 

less pronounced with increasing m. 

Up to what values of m should we expect the period two, maladaptive dynamics to 
dominate? In section VI we shall present a quantitative description of the low m-phase 
that addresses this point, but here we offer a useful semi-quantitative discussion. The 
back-of-the-envelope argument presented here is not quantitatively correct, but it does 
capture much of the nature of the djmamics that leads to the cross-over from the efficient 
phase as m approaches mg. First, consider two randomly chosen strategies of memory m. 
Suppose that following the occurrence of some string, ^, their relative rankings differ by 
one. We ask, what is the probability that, after a time of order 2™ (which is the mean time 
between successive occurrences of ^), they continue to differ by one in their rankings? 



16 



Roughly speaking, each of the other strings will be sampled once during a time of order 
2"*. Because the strategies are randomly chosen, each occurrence of one of the 
intervening strings will cause a change in their relative ranking of +1, -1, or 0, with 
probabilities 1/4, 1/4 and 1/2 respectively. Consequently, the change in their relative 
rankings over a time of order 2™ will be given by the excursion of a one-dimensional 
random walk of 2"* time steps, whose probability of moving to the left is 1/4, of moving 
to the right is 1/4, and of standing still is 1/2. The standard deviation of such a walk 

-1/2 

grows like (2"") , and the probability of ending up at the origin after 2™ time steps 
decreases like (2 ) . If there are N agents, then there will be about N/(2 ) agents the 
relative rankings of whose strategies will not change between successive occurrences of a 
given string. Now, absent any other djmamics, the number of agents voting for a given 
group at a given time will be of the order of N/2, with fluctuations of the order of N . 
The period two structure will dominate (so that the minority group on an even occurrence 
of ^ is the opposite of the minority group at the preceding odd occurrence of ^) if there is 
a sufficiently large number of agents, the relative rankings of whose strategies is 
unchanged. If so, then the deterministic period two dynamics will dominate over the 
order N random statistical fluctuations in the number of agents voting for a group at an 
even occurrence of ^. I.e., for the period-two dynamics to dominate, we must have 

HI "-^^^ 1/2 m 

N/(2 ) > N , or 2 /N < 1. This gives us a correct order of magnitude estimate for the 
phase transition between the efficient and inefficient phases, and also gives us one 
argument that leads, naturally, to the scaling variable, z=2™/N. 

The period two dynamics at small m is even more pronounced as s increases. In Fig. 16 
we plot a portion of the time series for m=2, but now with s=6. The bursting 
phenomenon is much more pronounced in this case, than in Fig. 15a. To understand this, 

m 

consider the case of infinite s (or, for practical purposes, s»2^ ). In this case, all agents 

m 

possess all possible strategies. (Note that even if s=2 not all agents will possess all 
strategies, since the strategies are drawn by the agents randomly with replacement, so that 
agents may have more than one copy of the same strategy.) At the first occurrence of 
some string, ^, each agent will have a subset of its strategies (at least two strategies), each 
of which has the largest number of points at that time step. For each strategy, t', in this 
subset which predicts a 1 following the string, ^, there will be another strategy in the 
subset, identical with t', but which predicts a following the string ^. Suppose each 
agent's decision about which group to join is determined by a coin flip, when its strategies 
rankings are tied. In that case, the population of group 1 will be near 50%, within random 
statistical fluctuations. Suppose the minority group at this time step turned out to be 
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group 0. Now consider the next occurrence of the string ^. Because each agent has all 
strategies available, it is easy to see that the top ranked strategy must be the one whose 
prediction following ^ is 0. To see this, suppose that the top ranked strategy was one 
whose response to ^ was 1. Since each agent has all strategies available, there is another 
strategy, identical to that one, but whose response to ^ is 0. Since the minority group 
following the last occurrence of ^ was 0, that strategy will have one extra point. Thus, on 
even occurrences of each string, all agents must join the group which was the minority 
group on the last odd occurrence of that string. But since all agents join that group (in our 
example, group 0), then the other group will turn out to be the minority group. This will 
even up the points among strategies that differ only in their response to string ^, so that 
the next time string ^ occurs, strategies which differ only in their responses to ^ will once 
again be tied in their rankings, and agents will choose which group to join randomly. It is 
not difficult to show that the variance of Lj in this infinite s limit is = N(N+l)/8, 

independent of m. Note that this result is obtained by first taking s to infinity. If s is 

2ni 

large, but much less than 2 , then a does depend on z, as shown in Fig. 11. 
R The transition region 

We will define the transition region in this system to be Zq < z < Zq, where Zq is the value 
of z at which a first becomes less than a in the RCG. It is over this range of z that there 
are dramatic qualitative changes in the behavior of the system It is, therefore, clearly 
advantageous to be able to study this region in detail. To that end we will introduce a 
new set of strategies that define strategy spaces with non-integer m. This will allow us to 
sample the behavior of the system at much finer intervals in z than would be possible 
were we restricted to only integer values of m. Aside from its use in studying the 
transition region, this generalization is very important for studying the robustness of the 
general phase structure of this system, to changes in the nature of the strategy space. This 
issue will be discussed in detail in a forthcoming publication^. In what follows, we will 
first study some useful quantities over a broad range of integer m. Later in this section, 
we will introduce the generalized strategy space for the purpose of studying the transition 
region in greater detail. 

We begin our discussion of the transition region by looking at the behavior of two useful 
quantities over a wide range of m. From the discussion in the preceding subsection, we 
know that the period two dynamics becomes less pronounced as z approaches Zc from 

below. To see this explicitly, we have plotted in Figs. 17 and 18 the probability, averaged 
over all m-strings, that the minority group following an even occurrence of a given string 
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is opposite of the minority group following the preceding odd occurrence of the same 
string (POED). These figures plot this quantity as a function of integer m for N=101 and 
two different values of s, s=2 and s=16. On the same graphs, we also plot the probability 
that the minority groups following two successive odd occurrences of a given string are 
different from each other (POOD). Refer first to Fig. 17. We note that the upper curve 
(POED) is close to one for small m, which is a reflection of the dominance of the period- 
two dynamics at small m. As m increases, POED decreases, reaching about 0.7 for m=5. 
This is what we expect: For m close to mc, not all the strategies maintain their relative 
rankings from one odd occurrence of a string to the subsequent even occurrence. 
Consequently, the populations of the minority groups associated with even occurrences of 
strings are generally closer to half the agents than is the case for smaller values of m. 
This means that the probability that the minority group following an even occurrence of a 
string is opposite that of the previous occurrence of the same string will no longer be 
close to one. As further confirmation of this picture, we plot in Fig. 19 a short segment of 
Li for N=101, m=5 and s=2. We see no large bursts in this time series as we saw, for 
example, in Fig. 15a. This, of course, is consistent with the fact that a is below that for 
theRCGform=5,N=101. 

Next, look at the lower curve in Fig. 17, POOD. This curve is always less than 0.5, rising 
to near 0.5 for integer m=5 and then decreasing. The reason that POOD is generally less 
than 0.5 is a consequence of the fact that the agents' strategies are fixed, and do not 
change over the course of the game. This can be understood by recalling that the 
strategies represent a fixed random sample of responses to different strings. Although, on 
average there will be as many strategies that respond to a given string as respond 1, for 
a given distribution of strategies to the agents, there will be statistical fluctuations in this 
number. Thus, absent any other dynamics, if the minority group is, say, 1 following a 
given string, then a priori, the probability will be greater than 0.5 that a subsequent 
occurrence of that same string will result in 1 again being the minority group. Of course, 
the period two dynamics in the low m-phase swamps this a priori expectation, and results 
in POED being close to 1. But since the period two djmamics does not affect POOD, that 
probability is still less than 0.5 in the low m phase. Note also that both POED and POOD 
decrease for m> mc, and approach 0.33 as m gets large. We shall explain this result in the 
next subsection. 

We now want to examine the region near the phase transition in more detail. To do that 
we need to generalize strategy space to a dimension of 2™ with non-integer m. To begin. 
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in a strategy of memory m, consider two strings of length m, say A and B, which differ 
only in their last bits (i.e. in the bit referring to the minority group m time steps earlier). 
In one of these strings (say A), replace the last bit, (0 or 1) with a "don't care", denoted by 
an asterisk (*). To avoid possible inconsistencies in the strategy table, it is then necessary 
to remove from the strategy table string B. Thus, for every asterisk inserted in this way 
into the strategy table, the dimension of the strategy space is reduced by 1, allowing m to 
take on non-integer values. An example of a strategy from such an interpolating space is 
shown in Fig. 20. In this way we can interpolate between a strategy space with a 
dimension 2^^, (M an integer) and one with dimension 2'^"\ To construct a strategy space 
with non-integer m, therefore, one randomly chooses a number of strings of length m, 
replaces the last bit with an asterisk, and eliminates the partner string. The remaining 
strings (some with asterisks, some without) define the dimensions of the strategy space. 
The value of m for such a strategy space is just log2[the number of different strings in the 
strategy table]. So, for example, the strategy shown in Fig. 20 is drawn from an 11- 
dimensional space yielding a value of m=log2[ll]=3.46. The dimension of the strategy 
space decreases as the number of asterisks used increases, and if each string contains as 
asterisk, then the dimension of the strategy space will have been reduced from 2^^ to 2*^ ^ 
A given set of strings is used for the left column of all the strategies of a given run. That 
is, in these games, all strategies look at the same set of information to make their 
decisions We have studied strategy spaces constructed in this way over a broad range 
of m and N, and have found that this procedure does produce results which smoothly 
interpolate between those presented above for integer m. 

Let us now take a closer look at the transition region using the method described above to 
interpolate between integer values of m. In the results of Figs. 21 and 22 discussed 
below, 16 runs were performed for each non-integer value of m. In each run, an 
appropriate sized strategy space was constructed by distributing asterisks randomly as 
described above (a different random distribution of asterisks for each run). For each run, 
random strategies in the strategy space so defined were assigned to the agents in the usual 
way. 

In Fig. 21 we show a as a function of m for 4<m<7, N=101 and s=2. In Fig. 22 we plot 
POOD and POED for the same parameters. In both these figures, we have divided both 



The very interesting situation in which not all strategies use the same set of information to make their 
predictions, will be discussed in Ref.9, above. 
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the ranges 4<m<5 and 6<m<7 into 16 bins each. The range 5<m<6 was divided into 32 
bins to show more detail. In Fig. 21a we show a for 16 different runs for each value of 
m. In Fig. 21b, we plot the average o averaged over all 16 runs for each value of m 
shown in Fig. 21a. First, in Fig. 21b, we see that the minimum in a does not occur at an 
integer value of m, rather, we see that a has a broad minimum extending from m=5.2 to 
m=5.7. This suggests that a reasonable estimate for mc would lie in this range. In Fig. 22 
we see that that POOD also has a broad maximum reaching 0.5 over a similar range, from 
m=4.8 to m=5.4. 

Look now at Fig. 21a. With the detail provided by this graph, we see an effect that was 
not evident when we studied only integer m. In this figure it appears that there are two 
distinct effects driving the phase change in the region of mc. There is a one dynamic 
which produces a broad distribution of o's for a given m, and which dominates for low 
m, and a second dynamic which produces a a's that lie in a narrow range, and dominates 
for larger m. The coexistence of the two djmamics in the region 4.8<m<5.5 is evident in 
this figure. This suggests that the phase change (which we have loosely referred to as a 
phase transition) may actually be accomplished by a smooth cross-over from one djoiamic 
to another. In analogy with physical systems, one may think of two local "energy" 
minima representing different states of the system, one associated with the strategy- 
efficient djaiamics of the low m phase, the other with the inefficient, cooperative 
dynamics of the high m phase. For any given parameter setting, the system prefers to be 
in a minimal energy state. For small m, the efficient minimum is also the global 
minimum, and so the system prefers to be in that minimum. For large m, the inefficient 
minimum is the global minimum, and so the system is preferentially in that minimum. 
As m passes through the transition region, these two minima smoothly change their 
relative values. When their depths are nearly the same, the system will sometimes occupy 
one minimum and sometimes another, as suggested in Fig. 21a for 4.8<m<5.5. That there 
is a phase change of the system through the transition region is clear. What its analytic 
structure is is not so clear. 



Two comments are useful here. First, it seems likely, at least as s^°°, that there is a singularity in at least 
some quantities, for example, Q(m) in Fig. 14. It may be, therefore, that for finite s the phase change is 
accomplished by a smooth crossover which becomes singular as s— >oo. Second, it is interesting to remark 
that the transition between the two phases is accompanied by a change in the nature of the system from 
essentially periodic to apparently non-periodic. In the low-m phase, for example, Li shows a distinct 
periodic structure. As m approaches the region near mc, the periods in Li become longer and more 
complex. In the inefficient, high m phase, there is no apparent periodicity in Li. 
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Now let us turn again to Fig. 22. First, we remark that, although not plotted on this 
graph, PEED, the probability that the minority group following two successive even 
occurrences of the same string are different from each other, forms the same curve as 
POOD. In particular, both PEED and POOD are 0.5 at mg. Second, it is interesting that 
POED is greater than 0.5, even at mc. Furthermore, PEOD, the probability that the 
minority group following an odd occurrence of a given string is different than that 
following the preceding even occurrence, is also about 0.6 at mc . These results paint a 
picture of cooperative dynamics in which responses to a given string are anti-correlated 
over times of order 2™, but are uncorrected over times of order 2'""^\ 

Given the argument that a priori, POOD and PEED should be less than 0.5, it is most 
remarkable that they reach the value of 0.5 at mc. The cooperative dynamics is evidently 
significant enough to overcome the a priori tendency for minority groups to be the same 
following any given string. On the other hand, it is not clear what the dynamical origin is 
of the fact that POED and PEOD are both greater than 0.5 at mc. One possibility is that 
this just indicates a remnant of the period two djmamics that dominated the efficient 
phase. In this case, the observed transition at mc may be more accurately thought of as a 
cross-over effect, as might be suggested by Fig. 21a. On the other hand, it is possible that 
these correlations are intrinsically bound up with the cooperative dynamics. We are 
currently agnostic about this issue, but its resolution is of significance. It is certainly 
important to understand the nature of the transition in as much detail as possible. But in 
addition, one may be interested in attempting to control the system to improve the degree 
of cooperation (lower o). Then, it may make substantial difference whether it is possible 
to identify a reasonably separable dynamics (e.g. the remnant of the maladaptive period 
two dynamics) which can be tuned down near mc to further enhance the agents' emergent 
cooperation. 

We also note in Fig. 22 fluctuations in POED, particularly near m=5. This is apparently a 
real detail, not a statistical effect, whose origin we do not fully understand. The smaller 
fluctuations in POED may also have dynamical significance. 

Our studies with non-integer m indicates that Zc is closer to 0.35 than to the value of 0.5 
that was deduced from the studies performed with integer m. We have examined a 



However, PEOD does differ from POED for lower m, as we would expect from our discussion of the 
period two dynamics in section IVA. 
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number of cases with different values of N. We find in all cases that the results for non- 
integer m smoothly interpolate between the results for integer m, that a /N still scales 
with z, and that the results still lie on a universal curve, but that the minimum of the curve 
is near Zc=0.35. 

To close our discussion of the transition region, we refer briefly to Fig. 18, in which we 
plot (for integer m) POED and POOD for the case N=101 and s=16. Here we see the 
same qualitative behavior as in Fig. 17, except that POED stays close to 1 throughout the 
efficient phase, and both POED and POOD approach a value close to 0.5 as m gets large. 
The behavior of POOD for small m is easy to understand on the basis of the discussion of 
the last subsection: If the agents have more strategies from which to choose, they will be 
more likely to find pairs of similar strategies for higher m, and so the period two 
dynamics will be more pronounced in the efficient phase. The asjmiptotic values of 
POOD and POED for large m will be explained in the next subsection. 

C Dynamics in the inefficient phase 

For z>Zc, the system is in a quite different phase. The most obvious hallmark of this 
phase, as shown in Fig. 7, is that the system is more effective at generating resources than 
would be the case in the RCG, but increasingly less so as z increases. For z near Zc, the 
agents are able to achieve maximal coordination and thus generate resources most 
effectively, but coordination evidently becomes less effective as z increases still further. 
To understand what happens, we will consider the information in G for various values of 
z. To illustrate, we will again use our canonical example of games with N=101 and s=2. 
In this section it will also be sufficient to consider only integer m. 

There are a number of different measures that we could use to characterize the 
information in G, including the entropy, but for our purposes a simpler measure is more 
appropriate. We are interested, in particular, in characterizing the information in G that is 
available to the strategies. Thus, we consider 

= ^I^KuJ = W I uj-^) (4.1a) 
and the sum over the squares of I(Um), 
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Lyi\uj = —y(P(i\uj--f 



(4.1b) 



The sum here is over all strings of length m. Clearly, the more P(llUm) differs from y^, the 
more information the string Um contains for predicting the next minority group. In Fig. 23 
we plot Im'^^^ (4.1b) as a function of m for the case N=101 and s=2. As we expect from 
our discussion of the dynamics of the efficient phase, Im is zero for m<mc. For m>mc Im 
is non-zero and increases for increasing m. As m increases, the system becomes 
increasingly inefficient with respect to the strategies, in that increasing amounts of 
information are left in G. 

It is not difficult to understand what information is contained in G. First, recall that the 
distribution of strategies to the agents represents a small random sample of all the 2 
possible strategies of memory m. Suppose we ignore any dynamics or cooperation among 
the agents, and just consider the probability that a 1 will follow any given m-string in G. 
In the absence of any coordination, each agent will choose randomly between its two 
strategies. Given a random choice between each of the agent's two strategies, the 
probability to have a 1 following a given m-string will not be exactly 0.5. The reason is 
that for any particular distribution of strategies, some agents will have the same response 
to a given string in both their strategies, thus skewing P(llum) away from 0.5, even in the 
absence of non-trivial dynamics. We can measure the skewness of the distribution of 
strategies by considering 



where v(0,0lum) and v(l,llum) are the number of agents both of whose strategies predict, 
respectively, and 1 following Um. The more positive (negative) ^(Um) is, the more likely 
it is that the minority group following Um will be 1 (0). 

Now consider the correlation between ^(Um) and I(Um). In Fig. 24 we plot Fm vs. m, 
where 




(4.2) 
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r„ =(/Km«j)-(/(«j)(^(Mj) 



(4.3) 



We see very clearly that Fni increases monotonically with m, indicating that the 
information left in G is increasingly correlated with the skewness associated with the 
agents' strategies. That is, as m increases G contains increasingly accurate information 
about certain features of the agents' strategies. 

Since the agents' choices are apparently less coordinated as m increases beyond m^, we 
expect that for very large m, the agents will choose between their two strategies randomly 
at each time step. If that is so, then we should be able to compute a good estimate of 
P(llUni) directly from v(0,0lum) and v(l,llum). If the agents choose between their two 
strategies randomly and with equal probability, then it is not difficult to see that the 
conditional probabilities become 



P,(ll«J = 2-'«£ 



ft)! 



■^A;!(A;-ft))! 



(4.4) 



where co=N-v(0,OIUm)- v(l,llum), 
a=N/2-v(0,0lum), 

and PR(llUm) are the conditional probabilities assuming that the agents choose between 
their two strategies randomly, independently, and with equal probability. 

In Fig. 25, we plot the average squared deviation between the observed conditional 
probabilities, P(llum), and PR(llUm) as a function of (integer) m. 



6 = 



1 y {P{l\uJ-P,{\\uJY 

2'"t' P"(X\uJ 



nl/2 



(4.5) 



Here we see that as m increases, P and Pr are increasingly similar, differing only by a 
few percent by the time m=8, and being nearly identical for m=ll. Thus, the information 
contained in G at large m is a simple reflection of the particular preferences embodied in 
the agent's strategies. 
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It may seem counter-intuitive that the system as a whole behaves more like the random 
choice game while the information in G increases. The point here is that the time series 
of minority groups is endogenous. It is created by the choices of the agents themselves. 
The fact that more information is left in G, just reflects that fact that the agent's choices 
are less coordinated, so that those choices use less of the information inherent in the 
strategies of the agents themselves. For this system, uncoordinated choices means that 
each agent chooses randomly, independently and with equal probability between his two 
strategies at each time step. Note, however, that this is not the same as the RCG in which 
each agent chooses or 1 randomly, independently and with equal probability at each 
time step. In the latter case G would be an ICD sequence with no information, (and the 
system would be in a mixed-strategy Nash equilibrium) but in the former case G contains 
significant information about the structure of the agents' strategic preferences (and the 
system is not in a Nash equilibrium). The existence of this fixed strategic preference 
structure is the reason that the system becomes increasingly inefficient (so that there is 
more information in G) as m increases, while at the same time the agents' choices 
become as uncoordinated and as random as possible. 

The fact that the agents' choices among their strategies becomes increasingly random as 
m increases also explains the behavior of POOD and POED in the large m limit seen in 
Figs. 17 and 18. It is not difficult to show that a random choice between two fixed 
strategies will yield a value of 0.33 for the probability that any two responses to the same 
string result in opposite minority groups. As s increases, the large m value of these 
quantities approaches 0.5. The reason is, that for larger s, an agent can choose from 
among more strategies in response to a given string. Thus, an agent will respond to a 
given string with either minority group with equal probability and with relative 
fluctuations of order s" . Consequently, POOD and POED should approach 0.5 as m 
increases for fixed large s, as we observe in Fig. 18. 

In general, the inefficient phase for larger s should have much the same qualitative 
structure as the s=2 example discussed above. We do expect, however, that the 
information in G for fixed m and N in the inefficient phase should decrease for larger s. 
The reason is that the information left in G reflects the conditional probabilities PR(llUm), 
and as s increases, these conditional probabilities should approach 0.5. At the same time, 
the agents look increasingly similar as s increases, for fixed m and N. But a 
homogeneous population cannot coordinate their strategy choices to produce large 
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minority groups. Therefore, we also expect to see less effective emergent coordination in 
the inefficient phase, as s increases, which is indeed what we observe in Fig. 11. 

V. Agent Wealth 

Given the interesting dynamics in the efficient and inefficient phases, it is natural to ask 
about the distribution of wealth (i.e. the number of points) to the agents. In a forthcoming 
publication,^ we will present a detailed analysis of agent wealth distribution, but here we 
wish to emphasize one important point. We ask, with what features of the agents' 
strategies is agent wealth associated. In particular, is there something intrinsic about 
strategies (or sets of strategies used by an agent) that are very successful, or do they gain 
their success only in the context of the other strategies being played in the game. 

To address this question, we define two distance measures. One is in an intra-agent 
distance, Dh(i) defined as the Hamming distance between the i* agent's two strategies. 
The second is a distance in "behavior space", Db(i) and may be understood to be the 
average behavioral distance of the i"^ agent from all other agents playing the game. In 
particular, if Ti^^(um) denotes the response (0 or 1) to the string, Um, of the j*** strategy of 
agent I, then 

DAi) = Y.K'('^J-T.''\uj\ (5.1) 
and 

(0 = m p(uJ\t!'' (m„ ) - rf («„ )| . (5.2) 

k,l u„ 

In Fig. 26 we plot accumulated agent wealth as a function of Dh and Db for games played 
with s=2 N=101 and various values of m. One first notes that for m<mc, agent wealth is 
quantized. This follows from the fact that the d5Tiamics in the low m-phase is periodic, 
an effect that will be discussed in more detail elsewhere^. But the point we wish to 
emphasize is the fact that for m<mc, agent wealth is strongly correlated with Dh. In this 
phase, the more similar an agent's two strategies, the wealthier the agent tends to be. 
Note also that for m<mc there is apparently no correlation between an agent's wealth and 
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Db. That is, in this phase it does not matter what all the other agents are doing. It only 
matters how similar an agent's two strategies are. 

For m>mc, the situation is quite the opposite. Now there is no correlation between agent 
wealth and Dh, but there is a strong correlation between agent wealth and Db. The further 
an agent is, on average, from all other agents in behavior space, the wealthier he tends to 
be. This is quite reasonable in a game in which points are rewarded for being in the 
minority. Why, then, is there no correlation between Db and agent wealth for m<mc? 

The reason is that for m<mc, there is no predictive information available to the agents' 
strategies and consequently no emergent coordination among agents' choices. For m>mc 
the agents use the real information in G to coordinate their choices. Those agents 
possessing strategies that allow them to behave maximally differently from other agents 
will more often find themselves in the minority group, and will accumulate more points. 
On the other hand, as we have demonstrated, for m<mc there is no real information 
available to the agents' strategies. Therefore, any attempt to use apparent information, 
and to make choices on the part of the agents, invariably leads to maladaptive herding 
behavior, which is not rewarded in the minority game. For m<mc, agents do best when 
then ignore, as much as possible, the misleading signals being sent by the collective 
behavior of the other players. In our game, a surrogate for ignoring those misleading 
signals is for an agent not to be able to make choices, and that is accomplished when the 
agent's strategies are as similar as possible. (If the agent's strategies were identical, the 
agent would effectively have only one strategy, and could not respond to the misleading 
signals of the collective by choosing to join different minority groups at different times, 
in response to a given string.) Although for m<mc the information in G available to the 
strategies is zero, in a very real sense, the information in G available to agents (i.e. 
information that extends over times greater than 2™) is negative, in that it leads to 
maladaptive decisions. 

VI. Mean-field description of the minority game 

In this section, we present a mean-field approximation for the behavior of the minority 

game. This description is most accurate for the behavior of the system for s extremely 

large (s>2 ). In addition, for smaller s, the description is applicable to the low m, 
efficient phase (z < Zc). In both the large s and low m regions, the key feature which 

allows us to write a mean-field approximation, is that the system response to an 
occurrence of a given m-string of minority groups is nearly independent of its responses 
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to the other m-strings. (Recall that it is this independence that is necessary for the period- 
two dynamics that dominates the low-m phase.) 

In formulating this mean-field description, we shall use the language of statistical 
mechanics, in which we will identify a set of independent variables and associate 
"energies" with various states of objects in our system. In particular, we shall identify m- 
strings of minority groups as variables in the system, and shall associate energies with 
strategies. The lower the energy of a strategy, the more likely it is to be chosen by its 
agent to be played. This is analogous to states of a statistical mechanical system, in 
which a system is more likely to occupy states with lower energy. In our case, there is 
simple relationship between the energy of a strategy and the number of points it has 
accumulated. One reason for introducing the language of statistical mechanics and 
ranking strategies by energy, is that this allows us to introduce a temperature into the 
system. This in turn lets us perform an interesting interpolation between the game as it 
was originally defined (the zero temperature limit), and a game in which the agents 
choose among their strategies randomly (the infinite temperature limit). This will be 
discussed elsewhere. 

Now, the primary objects of interest to us are the strategies. We will associate an energy 
with each strategy at each time step, T. Specifically, if a strategy predicts the next 
minority group correctly, its energy will be lowered by 1, and if it predicts incorrectly, its 
energy will be raised by 1. It is easy to see that a strategy's energy, E, is related to the 
number of points accumulated by that strategy. A,, by 2A,=T-E, where T is the time step of 
the game. 

The energy, Et, of a strategy, t, can be written in terms of the contributions to that energy 
associated with the response of the strategy to different m-strings of minority groups. 
Thus, consider a string, ^. Define the "string energy", e^, of ^ as follows:. Every time the 
string ^ is followed by a 1, e^ decreases by one and every time ^ is followed by a 0, e^ 
increases by one. Then, the cumulative contribution to the energy, Et, of any strategy, t, 
due to its responses to the string ^, is equal to e^, or -e^, depending on that strategy's 

response to the string, ^. I.e., 



Et = L^ p^(t)(-e^) 



(6.1) 
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where p^(t) is -1 if the strategy t predicts following the string ^, and is 1 if the strategy t 
predicts 1 following the string ^. 

We can now define our mean-field approximation of the minority game. The key to the 
simple treatment of the game in the infinite s limit (and in the low m-phase for finite s) is 
the independence of the response of the system to different strings, resulting in the simple 
period-two dynamics for each string. Thus in the mean-field theory the m-strings of 
minority groups are considered the independent degrees of freedom. 

The observation of the period two dynamics suggests that in this phase it is sufficient to 
allow all the string energies to take on only values of 0, 1, or -1, independently, and with 
probabilities 1/2, 1/4, and 1/4, respectively. Under these assumptions, we can easily 
compute P(E;n), the probability for a strategy of memory m to have a total energy, E, 
where n=2™. This is just given by the trinomial distribution, 

(n+|E|)/2 , 

^^^'""^^ k\ fc!(^-|£|)!(n + |£|-2)t)!4"-l-l2-l-l-^^ ' ^^'^^ 

Now, it is clear that the leading term in the variance of Lj in the low-m phase comes from 

the large bursts in this time series. These are the times at which the system responds to a 
string, ^, where the string energy, e^ is 1 or -1. I.e., it is those times at which there is an 

even occurrence of ^. Therefore, we will be interested in computing an estimate of the 

number of agents joining group 1 at an even occurrence of ^. For specificity, lets assume 
that e^=-l. We now calculate Pj, the probability that an agent joins group 1 in response to 
the string ^, if e^=-l. 

First, consider the case s=2. The probability that the lower energy of two randomly 
chosen strategies of memory m is E, is given by 

W^iE) = P(E-nl P(E;n) + 2Y,P{E';n)^\ ^^-^^ 

L E'>E J 

Now, if 2*° »1, we can approximate P(E;n) by a normal distribution: 

PiE;n).J^J=^] (6-4) 
\3m \ 3n J 

and the probability that the lower of the two energies is E becomes 
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W2(E;n)=2P(E;n)Q(E;n), (6.5) 
where 

Q(E;n) = ]p(E';n)dE'- (6-6) 

E 

This result can be generalized to the case of s strategies, so that the probability that E is 
the lowest energy among s randomly chosen strategies is given by 

WsiE-n) = -^Q\E-nr (^•'7) 

^E 

Now, given that e^ = - 1 , the probability that a strategy with energy E will choose group 1 
in response to string ^, is just given by 

P(E+l;n-l) 

P(E+l;n-l) + P(E-l;n-l) • ^^-^^ 

The reasoning is as follows: A strategy with n elements can be decomposed into two 

parts: The first, it's response to the string ^, and the second, the remaining n-1 elements. 

For a randomly chosen strategy, this latter part may be considered a random strategy with 

n-1 elements. There are two ways in which a strategy with n elements can achieve an 
energy E, given that e^ =-1. If the response of the strategy to ^ is 1, then the remaining n- 

1 elements of the strategy must contribute an energy of E+1 to the total energy of the 
strategy. Conversely, if the strategy's response to ^ is 0, then the remaining n-1 elements 
of the strategy must contribute an energy of E- 1 . Thus, the probability that a strategy with 
n elements and energy E chooses group 1, is just P(E-i-l;n-l), normalized by the total 
probability to find a strategy of energy E. 

Consistent with our restriction that the e^ independently takes on only values 1,-1 and 0, 
(i.e., because of the period two dynamics) E~0(n^^^), if n»l. In that case, expression 
(6.8) can be approximated as 

P(£ + l;n-l) (6.9) 

P(E + l;n-l)+P(E -l;n-l)" 2 3n 



We can now estimate pi, the probability that an agent will choose group 1, given that e^=- 
1. It is 
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2 

where 



\W,(E;n)dE = ^^l- 



- 1 



f 

X(s)=lxdxj-j=le 



dx 



2 dy 



(6.10) 



(6.11) 



The function x(s) is plotted in Fig. 27. Note that %(s) is zero for s=l, and varies relatively 
slowly for s>5. 

Now, the average number of agents joining group 1, given that e^=-l is just piN. So long 
as the fluctuations in this number are small enough, the number of agents choosing 1 will 
always be greater than N/2, and the period-two dynamics will obtain. This will certainly 
be the case if N%(s)2/2'"=%(s)2/z»l. Thus, under these circumstances, a majority of 
agents will choose group 1 when e^=-l, and the variance in the attendance from N/2 
given that e^= -1 will be 

'-f^(-i)".f5^^-<«. (6.12) 



This result is a good approximation as long as the second term in (6.12), N%(s)2/2'" 
=X(s)2/z»l, or z«l. For z > 1, this picture breaks down. The e^ take on values other 

than 0, 1 and -1, and the response of the system to different m-strings lose their 

independence. Since this picture, and the approximation in equation (6.9) breaks down 
when z > 1, we expect this to be the point at which the phase transition occurs, consistent 

with Fig. 7. Note also that the form of the second term in equation (6.12) is consistent 
with our earlier argument for the position of the phase transition. 



VII. Summary and Discussion 

A. Summary 

In this paper we have presented an overview of the rich and complex behavior that very 
simple minority games display. Our main results are: 

1. As a function of the size of the strategy space from which agents draw their strategies, 
minority games display two phases separated by a clear transitional region. The first 
(small strategy space) is an informationally strategy-efficient phase in which no 
predictive information about the next minority group is available to the agents' 
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strategies. The second (large strategy space) is a phase in which information is 
available to the agents' strategies. 

A. For small strategy spaces the system does poorly at generating resources. The 
dynamics in this phase is dominated by a simple period two dynamics for 
which it is possible to write a mean-field theory. This dynamics leads to 
agents' choices that are maladaptive resulting in worse than random 
performance. The system is also strategy-efficient, in that there is no 
information available to the agent's strategies that can be used to determine 
which group, at any given time step will be the minority group. 

B. For larger strategy spaces the agents can coordinate their decisions to produce 
better than random generation of resources. As the strategy space increases in 
size, agent coordination becomes less effective and, for very large strategy 
spaces, over-all system performance approaches that of a random choice 
game. At the same time, the information in the data set that the agents use to 
determine their strategy choices becomes greater as the size of the strategy 
space increases. For large strategy spaces, this information directly reflects 
the probability structure of choices embodied in the agents' strategies. 

C. The best resource generation is obtained in the transition between the two 
phases, when the dimension of the strategy space is on the order of the 
number of agents playing the game. 

2. For different numbers of agents, and different sizes of the strategy space, this general 
phase structure is a function only of the ratio of the dimension of the strategy space 
divided by the number of agents. 

3. Agent wealth is correlated with different strategy characteristics in the two phases. 

A. In the small strategy space, maladaptive phase in which the agents exhibit 
herding behavior, agents whose strategy sets are composed of very similar 
strategies do best, without regard to the nature of those strategies, or to the 
nature of the strategies of the other agents. 

B. In the large strategy space phase, agents whose strategies are maximally 
distant in behavior space from all other strategies do best. Thus, the value of 
a given strategy is dependent on details of that strategy in the context of all 
other strategies in the game. 
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4. All of this structure depends on the agents having a small to moderate degree of 
adaptivity. In the case in which there is only one strategy per agent, this structure 
disappears, and the game becomes trivial and periodic. On the other hand, as the 
number of strategies available to each agent increases beyond two, coordination 
becomes more difficult. The general phase structure still obtains, but the degree of 
coordination diminishes and the system generally performs more poorly at the 
transition and in the inefficient phase than it does when each agent has fewer 
strategies (but still more than one). 

B. Discussion 

The very rich behavior seen in these simple minority games is quite remarkable, as is its 
apparent robustness. Although not reported in detail in this paper, we have also studied 
the robustness of the general phase structure of these games to changes in the information 
set used by the agents' strategies. Of course, the interpolated strategy spaces introduced 
to study the detailed behavior in the transition region do represent one significant 
alteration in the information set available to the agents. The fact that the results smoothly 
interpolate between the strategies spaces with integer m speaks to the generality of our 
results. But in addition, we have found that when other kinds of information is given to 
the agents — for example, more specific information about the number of agents in a 
group as a function of time — the general phase structure of the system remains the same. 
There is a single phase transition between a poorly performing strategy-efficient phase, 
and a better performing strategy inefficient phase. Moreover, the transition still occurs 
when the ratio of the dimension of the strategy space divided by the number of agents in 
the game is a number is of order one^. 

Nevertheless, there are a number of effects that could alter the system's dynamical 
structure significantly. One feature of the games described in this paper is that all of the 
agents are looking at the same signals. That is, the strategy space is the same for all 
strategies. It may be that the existence of a set of common signals on which the strategies 
base their predictions allows the agents to coordinate their strategy choices in a relatively 
simple way. One straightforward way to change this condition, is to allow different 
strategies to use different memories. We are currently studying such games to assess the 
importance of a universally shared strategy space. 

Another very important issue to address is the role of evolution in such games. Although 
the systems discussed in this paper were adaptive, they were not evolutionary, in the 
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sense that the agents' strategies were fixed from the beginning of the game. It is unclear 
how the system will behave when evolution is introduced and the agents are allowed to 
change their strategies. It does seem likely, however, that the results will depend on some 
of the characteristics of the way evolution is introduced into the system. In particular, it 
is clear that if evolution takes place on a very fast time scale (significantly shorter than 
2"*) then the system may devolve into random behavior, since the agents will not have 
time to learn and coordinate their strategy choices with the other agents. For evolution on 
a slower time scale, though, the outcome is unclear. One intriguing possibility, however, 
is that the system would evolve to a set of strategies with memory near mc. 

A natural question to ask about our results is their relation to Nash equilibria. The 
minority games have many Nash equilibria. For example, one Nash equilibrium is a rule 
in which the first (N-l)/2 agents always join group 0, the last (N-l)/2 agents always join 
group 1, and the remaining agent alternates between the two groups. There are clearly 
many variations on this rule that are also Nash equilibria. The mixed strategy rule in 
which each agent randomly, independently and with equal probability chooses between 
the two groups (i.e. the RCG) is also a Nash equilibrium. However, it does not appear 
that the states achieved by the dynamics of the games described here are Nash equilibria. 
Even for very large m, where the system-wide behavior is that of the RCG, the system is 
not in a Nash equilibrium. Although the agents do choose between their strategies 
randomly, they do not choose the minority groups randomly. It would be possible, 
therefore, for some agent to choose some other deterministic ordering of choices for his 
strategies and increase his wealth. That this is possible, in principle, follows from the fact 
that G is not an IID sequence. Thus, there is information that can be exploited to improve 
an agent's wealth. However, the dj^amics of this game do not allow the agents to exploit 
that information. These observations, of course, are just special cases of the general idea 
that dynamics in an adaptive system can prevent the system from achieving equilibria, 
and in particular, Nash equilibria. In our opinion, they are important indicators of the 
need to seriously consider dynamics in the analysis of economic and other complex 
adaptive systems, and not rely solely on equilibrium analyses. 

It is also interesting to consider again the nature of the strategies and their success in these 
games. In particular, consider the inefficient phase in which the strategies in play 
represent only a small sample of the total number of strategies in the strategy space. Here 
some strategies perform better at predicting minority groups than others. But a priori 
there is nothing special about any particular strategy. They are all just randomly 
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generated strings of and 1. Nevertheless, in the context of the other strategies 
represented in that particular game, certain strategies perform better than others. A 
necessary condition for the emergence of this ordering is the fact that the strategies in 
play represent only a small sample of the total strategies possible. Indeed, increasing the 
number of strategies in play, either by increasing N or s sufficiently, puts the system into 
a region in which no strategy performs better than any other. Thus, there is a sense in 
which there is an emergent meaning to specific a priori random strategies, which 
meaning emerges contextually in an environment representing a small random sample. 
We believe that a sparsely sampled configuration space may be a general condition for 
the emergence of meaning. The simplest example of this is the observation that it is 
always possible to find structure and patterns in a finite string of random numbers. As the 
length of the string grows, however, finite length patterns lose their significance. The 
kinds of meaning that emerge in different contexts, and the criteria for what constitutes a 
"small" random sample will certainly depend on the djmamics associated with the system. 
Nevertheless, we believe that a sparse sampling of the configuration space of the system 

21 

may well be a necessary condition for the emergence of meaning . 

This system also has some characteristics that are reminiscent of a spin-glass in statistical 

22 

mechanics. In a spin-glass , the interactions among particles are mediated by a set of 
fixed, randomly distributed pair-wise interactions, which may differ in sign, some being 
ferromagnetic (causing spins to align in the same direction) and others being anti- 
ferromagnetic (causing spins to align in opposite directions). In general the system may 
not be able to satisfy all these tendencies for a given distribution of interactions, while at 
the same time achieving a low energy. This is the phenomenon of frustration, and leads 
to a very rich behavior of the system. In our game, the strategies that are distributed to 
the agents typically represent a small random sample of all possible strategies. They are 
fixed for the duration of the game, and they endow agents with strategic preferences 
which, in general, cannot all be satisfied, while at the same time achieving an optimal 
utilization of resources. Much of the rich behavior of our game is intimately bound up 



Of course, this discussion can be put into the larger context of symmetry breaking. In that case, one can 
suggest a number of mechanisms that might break a pre-existing symmetry and cause associations and 
meaning to emerge. Which of these mechanisms can be subsumed under the rubric of a sparsely sampled 
space is a semantic question. But in the context of many social and biological systems, a sparsely sampled 
configuration space (interpreted more narrowly) may turn out to have been a precondition for the 
emergence of meaning. 

See, for example. Spin Glasses, K. Fischer and J. Hertz (Cambridge University Press, Cambridge, 1991); 
Spin Glass Theory and Beyond, M. Mezard, G. Parisi, and M. Virasoro (World Scientific, Singapore, 
1987). 
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with the frustration associated with highly constrained agent strategic preferences. It will 
be interesting to study the ways in which frustration effects change if evolutionary 
dynamics are introduced into the system, so that agents are allowed to alter their strategies 
in response to selective pressure. The general djmamic of frustration is likely to be a very 
important one in the study of adaptive competition and evolution in biological and social 
systems^^, and deserves more intensive study in that context. 

Finally, it is clear that our work has implications for the study of a wide range of complex 
adaptive systems. The apparent generality of the phase structure suggests that this may be 
a persistent feature of a large number of specific systems, even when more realistic 
details are included. At the very least, this underlying phase structure is likely to be a 

24 

base upon which the behavior of more complicated specific systems are built."" In 
addition, we believe our work raises questions about the underlying epistemology of 
complex systems. There are currently few, if any, established general principles for the 
emergent behavior of complex systems. In fact, we do not even know what the proper 
questions are to ask about such systems. For example, we do not know, in general, what 
features of systems are specifically dependent on the details of the systems, and what are 
more generic. In our simple model, for instance, one might not have expected that a in 
the efficient phase would have been so strongly dependent on the particular distribution 
of strategies to the agents. This absence of guiding principles, of course, has profound 
implications for the veracity of conclusions that can be drawn from more realistic models 
of specific systems. We believe that an epistemology for complex systems will emerge 
from the aggregation of insights garnered over time from a range of research in this area. 
The investigation of very simple prototj^ical systems, such as the one presented in this 
paper, has much to contribute to the evolution of an epistemology of complex systems. 



See, for example, The Economy as an Evolving Complex System, P. Anderson, K. Arrow and D. Pines 
(eds.), (Addison- Wesley, Reading MA, 1988); The Complexity of Cooperation, R. Axelrod, (Princeton 
University Press, Princeton NJ, 1997). 

In this regard, it is interesting to note that the SFl artificial stock market described in W.B. Arthur, J. 
Holland, B. LeBaron, R. Palmer, and P. Tayler, Santa Fe Institute Working Paper 96-12-093 (1996) 
describes two phases as a function of the rate at which agents are allowed to evolve their strategies. The 
SFI artificial market model is much more complex than the games described in this paper. However, it may 
be that the evolution rate in the SFI market model can be thought of as a proxy for the size of the strategy 
space available to the agents, and that the two phases seen there are reflections of the underlying dynamics 
we have discussed in this paper. 
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Figure Captions 
Fig. 1. An example of an m=3 strategy. 

Fig. 2. a, the standard deviation of the time series of the number of agents joining group 
1 (Li) as a function of m, for N=101 agents, each with s=l strategies. Each dot represent 
an independent run; 32 runs of 10,000 steps each were performed for each m. The 
horizontal line (at o=5) is the value of o for the random choice game (RCG) described in 
the text. 

Fig. 3. a as a function of m for N=101 agents, each with s=2 strategies. Note the broad 
spread in values of a from the runs done with m<6. 

Fig. 4 a) A histogram of the conditional probabilities P(lluk) with k=m=4. There are 16 
bins corresponding to the 16 possible combinations of four O's and I's. The bin numbers, 
when written in binary form, yield the strings Uk. b) A histogram of the conditional 
probability P(lluk) with k=5 for the game played with m=4. There are 32 bins 
corresponding to the 32 possible combinations of five O's and I's. 

Fig. 5. A histogram of the conditional probability P(lluk) with k=m=6. There are 64 bins 
corresponding to the 64 combinations of six O's and I's. 

Fig. 6 a) a as a function of N, for s=2, and for m=3 and m=16, on a log-log scale. Note 
that for m=3, when the system is in an informationally efficient phase, oocN, while for 

1/2 

m=16 (in the information inefficient phase), o<xN . b) The relative spread in a, Aa/a, 
as a function of N, for s=2, and for m=3 and m=16, on a log-log scale. Note that in both 
cases Aa/a is independent of N, indicating that in the information efficient phase (m=3), 

1/2 

AaocN, while in the information inefficient phase (m=16), AaocN . 

2 m 

Fig. 7. a /N as a function of z=2 /N for s=2 and for various values of N and m, on a log- 
log plot. 

Fig. 8. a as a function of m for s=6 and N=101. Compare to Fig. 3 (s=2). 
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Fig. 9a) a as a function of N, at fixed s=6, for m=3 and m=16, on a log-log scale. As in 
Fig. 6a (s=2) for m = 3 (in the informationally efficient phase) G<xN, while for m=16 (in 

1/2 

the information inefficient phase), o<xN . b) The relative spread in o, Aa/a, as a 
function of N, for s=6, and for m=3 and m=16, on a log-log scale. As in Fig. 6b (s=2), 
Aa/a is independent of N for m=3 and m=16, indicating that in the information efficient 

1/2 

phase (m=3) Aao^N, while in the information inefficient phase (m=16), Aao^N . 

2 m 

Fig. 10. a /N as a function of z=2 fN for s=6 and for various values of N and m, on a 
log-log scale. Compare to Fig. 7. 

2 m 

Fig. 11. a /N as a function of t^=7Jz{s), for a range of values of s, where z=2 /N (as in 
Figures 7 and 10), and z (s) is the value of z at which a is a minimum for a given s. Thus 

2 

the minima of a /N in this plot are at ^ =1. 

Fig. 12 a) A histogram of the conditional probabilities P(lluk) with k=m=4 and with s=6. 
Compare to Figure 4a, which shows the histogram for the s=2 case, b) A histogram of 
the conditional probability P(lluk) with k=5, m=4 and s=6. Compare to Figure 4b, which 
shows the results for the s=2 case. 

Fig. 13. A histogram of the conditional probability P(lluk) with k=m=7 and with s=6. 
Compare to Figure 5, which shows a histogram of a game in the inefficient phase for the 
s=2 case. 

Fig. 14. Q(m), the mutual information in the time series of minority groups, as a function 
of ^=z/z^(s), on a semi-log plot, for values of s=2 and s=16. See equation (3.1) for the 

definition of Q(m). 

Fig. 15 a) A representative short segment of a typical time series of Li, the number of 
agents joining group 1, for the case of N=101, m=2 and s=2 (in the informationally 
efficient phase). The zero of the vertical scale means a population in group 1 of 50 
agents. The horizontal dashed lines show the range of one standard deviation for the 
N=101 random choice game (RCG). The circles represent those times at which the 
previous two minority groups were 01, as explained in the text, b) A short segment of Li 
for a game in the inefficient phase, m=8, N=101 and s=2. The zero of the vertical scale 
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again represents a population in group 1 of 50 agents, and the horizontal dashed lines 
show the range of one standard deviation for the N=101 random choice game (RCG). 

Fig. 16. A short segment of Li for m=2, N=101 with s=6 strategies per agent. The zero of 
the vertical scale represents a population in group 1 of 50 agents, and the horizontal 
dashed lines show the range of one standard deviation for the N=101 random choice 
game (RCG). 

Fig. 17. POED and POOD as a function of m, for N=101 and s=2. POED and POOD are 
defined in section IVB in the text. 

Fig. 18. POED and POOD as a function of m, for N=101 and s=6. 

Fig. 19. A short segment of Li for m=5, N=101, and s=2, (i.e., in the transition region, 
just below mc). 

Fig. 20. An example strategy from an 11 -dimensional strategy space. This strategy space 
is produced by inserting five "don't care" sjmibols (*) in the left-most column of a subset 
of length m=4 strings. This 1 1 dimensional strategy space yields an interpolated value of 
m = log2[ll]=3.46. 

Fig. 21. a) a as a function of interpolated values of m, for 4 < m < 7, for N=101 and s=2. 
The ranges 4<m<5 and for 6<m<7 are each divided into 16 bins, and the range 5<m<6 is 
divided into 32 bins. There are 16 independent runs for each (integer and non-integer) 
value of m. Each independent run includes a different random placement of asterisks (*) 
to define the strategy space, as well as a random distribution of strategies to the agents, 
b) The average of the o's from fig. 21a over the 16 runs for each bin as a function of 
interpolated m for 4 < m < 7 and for N=101 and s=2. 

Fig. 22. Average values of POED and POOD for interpolated values of m, 4 < m < 7, for 
N=101 and s=2. The bin structure is the same as in Fig. 21. Each point represents the 
average of the POED and POOD over the 16 runs for each bin. 

Fig. 23. I^^^m, the square summed information available to the strategies in the time series 
of minority groups (G), defined in eq. (4.1b), as a function of integer m for N=101, s=2. 
Values for 8 independent runs are plotted for each m. 
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Fig. 24. Fm, (eq. (4.3)) the correlation between Im (the information in G available to the 
strategies, defined in eq. (4.1a)) and (the skewness in the distribution of strategies, 
defined in eq. (4.2)), as a function of integer m for N=101, s=2 Note that as m increases, 
there is increasing correlation between the information left in G and the skewness of the 
agents' strategies. 

Fig. 25. (t)m (eq. (4.5)), the average squared deviation between the observed conditional 

probabilities P(llUm) and calculated estimates, PR(llUin) defined in (4.4), as a function of 
m, for N=101 and s=2. ^m=0 indicates the conditional probabilities P and P^ are the 

same. 

Fig. 26. Relationship between agents' inter-agent strategy distances, (Db), and intra-agent 
strategy distances, Dh, and their accumulated wealth in typical runs with N=101 and s=2, 
for various values of m. Dh and Db are defined in eqs. (5.1) and (5.2), respectively, a) 
m=3, Db. b) m=3, Dh. c) m=4, Db. d) m=4, Dh. e) m=5, Db. f) m=5, Dh. g) m=6, Db. 
h) m=6, Dh. i) m=7, Db. j) m=7, Dh. k) m=8, Db. 1) m=8, Dh. 



Fig. 27. %(s), defined in eq. (6.11), as a function of s. 



